Skip to content

docs: correct the certification guides against current libraries - #408

Merged
marc0olo merged 8 commits into
mainfrom
docs/certification-accuracy
Sep 28, 2026
Merged

marc0olo merged 8 commits into
mainfrom
docs/certification-accuracy

Conversation

@marc0olo

@marc0olo marc0olo commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Summary

Both certification guides had claims that are wrong on current releases. Every claim was checked against primary sources (the crates, @icp-sdk/core 6.1.0, ic-gateway, the interface spec, icp-cli v1.5.0 cli.md), and every code block was compiled and run against a local network. The skill side of the same fixes is dfinity/icskills#407.

Wrong, now fixed:

  • "Certified data is cleared on upgrade" (5 places): it survives upgrades (abstract-behavior.md, confirmed locally). What is lost is a heap tree, so Rust rebuilds it in post_upgrade, and Motoko's CertTree.Store needs no hook.
  • Header name: IC-Certificate-Expression is really IC-CertificateExpression (gateway spec, crate constant, served headers).
  • icp canister call … get without --query: icp-cli sends an update call by default, so no certificate comes back.
  • Client verification code: it did not type-check against @dfinity/certificate-verification 4 (Uint8Array, not ArrayBuffer). lookupResultToBuffer treated Unknown as absent; it now switches on the lookup_path status. It also accepts bindgen's undefined for an empty opt field, which it rejected as a false "key is absent" failure.
  • The "custom HTTP client" case needs @dfinity/response-verification, not certificate-verification. This is now a "who verifies what" table (HTTP gateway, update calls, Candid queries, raw hosts).
  • The ic-asset-certification example did not compile (missing candid, 404 needs StatusCode). Its uncertified 404 was rejected by the gateway; it now uses a certified 404.html fallback.
  • Rust example: it lacked export_candid!(), so icp deploy failed. It now uses ic-certification 4 / ic-cdk 0.20.
  • The single-value Motoko example never certified its initial value, so a query before the first write failed verification (certified data starts empty).
  • Motoko CertTree example did not compile (CertTree.Ops must be transient). The deprecated postupgrade hook is removed, and remove becomes delete to match the test commands.
  • allow_raw_access: false: redirects with 308, but raw.icp.net lands on <id>.icp0.io, and only mainnet raw hosts are recognized.
  • Asset canister certified headers: also Cache-Control (with max_age) and Content-Encoding.
  • "Boundary node" wording: the intro linked the API boundary nodes, which verify nothing; both pages now say the HTTP gateway.
  • Root key: shouldFetchRootKey is replaced by the ic_env cookie / icp network status --json.
  • Links: the js.icp.build link is dropped (that site does not document this package), and certified-counter (dfx, fetchRootKey) is replaced by motoko/cert-var.

After review:

Structural note: the asset and Rust examples stay inline even though they exceed the 30-line guideline, as before. They could move to dfinity/examples with #region markers.

Fixes claims that were wrong or outdated: certified data survives upgrades,
the certification header is IC-CertificateExpression, icp canister call needs
--query to return a certificate, and raw-access redirects land on icp0.io.
Code examples now compile and verify against ic-cdk 0.20, the 4.x
certification crates and @dfinity/certificate-verification 4.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The raw-host verification guidance and Rust upgrade example remain technically incorrect.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 4 Medium severity · 1 Low severity

Open (5)
What changed in this PR

Updates certification guides to match current ICP libraries and runtime behavior.

Changes:

  • Corrects HTTP certification headers, gateway behavior, and client verification.
  • Updates Rust, Motoko, TypeScript, and icp-cli examples.
  • Clarifies certified-data behavior across upgrades.
File Description
docs/​guides/​frontends/​certification.md Updates HTTP certification and verification guidance.
docs/​guides/​backends/​certified-variables.md Revises implementations, dependencies, and upgrade behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/guides/backends/certified-variables.md Outdated
Comment thread docs/guides/backends/certified-variables.md Outdated
Comment thread docs/guides/frontends/certification.md Outdated
Comment thread docs/guides/frontends/certification.md
Comment thread docs/guides/frontends/certification.md Outdated
Scope the gateway guarantee to verifying hostnames, describe what each
certification header carries, name every context certified_data_set
allows, and state that the Rust example keeps its tree on the heap only.
@marc0olo

Copy link
Copy Markdown
Member Author

Feedback addressed:

  • The Rust post_upgrade comment now states that the example keeps its tree on the heap only.
  • certified_data_set contexts now match the interface spec (every replicated context; traps in a query).
  • The gateway guarantee is scoped to verifying hostnames.
  • The two certification headers are described accurately.
  • The raw-host wording is now "forwards it without checking it". Not changed: the verifyRequestResponsePair row, because raw responses keep their certificate (verified on mainnet and in the gateway source); upstream fix in docs: "on raw the gateway simply discards it" reads as if the certificate is removed certified-assets#138.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The replicated-context claim and raw-host root-key guidance remain inaccurate or unsafe.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
Resolved since last review (5)

Comment thread docs/guides/frontends/certification.md Outdated
Comment thread docs/guides/backends/certified-variables.md Outdated
… root key

certified_data_set is allowed in init, upgrade hooks, updates, reply and reject
callbacks and system tasks, not in cleanup callbacks or any query. The ic_env
root key is only trustworthy on a verifying hostname.
@marc0olo

Copy link
Copy Markdown
Member Author

Feedback addressed:

  • certified_data_set now lists the spec's allowed contexts explicitly, including pre_upgrade and the low-Wasm-memory hook, and excludes cleanup callbacks and queries.
  • The ic_env root key is scoped to verifying hostnames; a client verifying raw responses needs an independently obtained key.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The single-value verifier does not accept the optional certificate type returned by its Motoko getter.

Review effort: Balanced
Findings: None

Resolved since last review (2)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Handle optional certificate before Certificate.create

docs/​guides/​backends/​certified-variables.md:331

This helper is presented as the verifier for the Motoko getter above, but that getter returns certificate : ?Blob. Its generated response therefore has an optional certificate, so it cannot be passed to this { certificate: Uint8Array } parameter without an unchecked unwrap. Accept the nullable field and fail explicitly when the getter was not executed as a query before calling Certificate.create.

@marc0olo

Copy link
Copy Markdown
Member Author

Feedback addressed:

  • Both verification helpers accept the ?blob certificate that the Motoko getters return (Uint8Array | null), and fail with a clear message when it is null (the getter was not called as a query). Verified against live Rust and Motoko canisters, including the null case.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The TypeScript helper does not accept the Motoko example’s Uint8Array value without normalization.

Review effort: Balanced
Findings: None

Previously missed (2)

In code that hasn't changed since last review

Medium severity CertTree blob getter type mismatches helper input

docs/​guides/​backends/​certified-variables.md:285

This helper is presented after both implementations, but the Motoko CertTree getter returns value : ?Blob, which bindgen exposes as Uint8Array | null; the declared string | null input therefore does not type-check with that example. Normalize the blob value before comparing it, or explicitly scope the helper to the Rust getter.

Low severity Clarify Motoko response values require blob decoding

docs/​guides/​frontends/​certification.md:274

“The response can be passed as is” is not true for the Motoko CertTree response linked from this section: its value : ?Blob becomes Uint8Array | null, while the helper accepts string | null. Clarify that only the certificate and witness already match, and that blob values must be decoded first.

@marc0olo

Copy link
Copy Markdown
Member Author

Feedback addressed:

  • The witness helper accepts both getters' responses as returned: the Rust opt text value and the Motoko CertTree ?Blob value (Uint8Array | null, decoded as UTF-8). The frontend guide no longer claims the response passes "as is" without saying which fields.
  • Verified with the raw responses of both live canisters, including an absent key and a tampered value.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The single-value verifier lacks the documented certificate freshness check, permitting stale replay.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread docs/guides/backends/certified-variables.md
@icp-sdk/bindgen maps opt record fields to optional properties, so an
absent key's value arrives as undefined and the helpers rejected a valid
proof of absence. Also drop step numbers the frontend page never defines,
say the HTTP gateway (not the boundary node) verifies HTTP responses,
and note that agent.rootKey is nullable.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The raw-gateway guidance conflicts with the currently synced static-site documentation until PR #409 lands.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Low severity Update after #409 to avoid contradictory certification guidance

docs/​guides/​frontends/​certification.md:50

The PR description says #409 has already synced this wording, but the current target still pins certified-assets to v0.4.0, and static-site/how-it-works.md:68-71 says the raw gateway “discards” the certificate. Until #409 is merged and this branch is updated, the two certification guides give contradictory behavior for the same raw response. Please make this PR depend on the synced change or rebase it after #409 lands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The updated guidance is consistent with the repository specifications, linked documentation, and current examples.

Review effort: Balanced
Findings: None

@marc0olo
marc0olo marked this pull request as ready for review September 28, 2026 09:48
@marc0olo
marc0olo requested a review from a team as a code owner September 28, 2026 09:48
marc0olo added a commit to dfinity/icskills that referenced this pull request Sep 28, 2026
…x broken examples (#407)

The `certified-variables` skill is brought up to date with
[`@dfinity/certificate-verification`
4.0.0](https://www.npmjs.com/package/@dfinity/certificate-verification/v/4.0.0),
which peers `@icp-sdk/core` ^6, and with the current Rust and Motoko
libraries. Every code block was compiled, deployed to a local network
(icp-cli 1.5.0) and exercised end to end, and several were broken on
`main`.

**Broken on `main`, now fixed:**
- **Frontend verification did not type-check** against
certificate-verification 4 / core 6. The parameters are now
`Uint8Array`, not `ArrayBuffer`. `lookup_path` returns a `LookupResult`,
not bytes, so the old `if (!leafData)` check never fired. The new code
switches on `Found`/`Absent` and rejects `Unknown`. Treating `Unknown`
as absent would let a replica hide a real value behind a witness for a
different key (new pitfall 6). The helpers also accept
`@icp-sdk/bindgen`'s output as returned, where an empty `opt` field is
`undefined`, not `null`.
- **The Motoko `CertTree` example did not compile.** `let ct =
CertTree.Ops(...)` fails with `M0131` in a persistent actor and needs
`transient`. The deprecated `postupgrade` hook is dropped.
- **The Rust example failed `icp deploy`.** It lacked
`ic_cdk::export_candid!()`, so the Rust recipe failed with
`get_candid_pointer`.
- **Every `icp canister call … get` was wrong.** icp-cli sends an update
call unless `--query` is passed, so the Rust getter trapped and the
Motoko getters returned `certificate = null`.
- **The upgrade pitfall was wrong.** Certified data **survives**
upgrades (`abstract-behavior.md:2690-2693`, confirmed locally). What is
lost is a heap tree, so Rust must rebuild it and re-set the hash, while
Motoko's `CertTree.Store` needs nothing.
- **The HTTP certification snippet was incomplete.** It had no
`http_request` handler, no witness header, and no
`IC-CertificateExpression` header in the certified response. The last
one makes `ic-http-certification` 4 return
`CertificateExpressionHeaderMissing`. It is replaced by a complete
minimal canister, moved to `references/http-certification.md`.

**Restructured for agents.** The skill now opens with **"Who Verifies
What"**, which answers whether client-side verification is needed and
with which library:

| Response | Verified by |
|---|---|
| HTTP via `<id>.icp.net` or a custom domain | the gateway |
| HTTP via `raw` or your own client | nobody; use
`@dfinity/response-verification` |
| Update call through an actor | the agent |
| Candid query | only the answering node's signature; use certified data
plus `@dfinity/certificate-verification` |

The ICRC-3 tip certificate is covered as a use case.

The body shrinks from 482 to about 370 lines, and the size warning is
gone:
- Overlapping pitfalls are merged.
- The conceptual diagram is dropped.
- The deploy walkthrough and "Verify It Works" become a two-command
check.
- The HTTP material moves to a reference.

**Updated:**
- **Rust:** `ic-cdk` 0.20, `ic-certification` 4 (same `RbTree` API as
`ic-certified-map`, and what the official examples use),
`ic-http-certification` 4.
- **New no-witness frontend path** (`Certificate.create` +
`certified_data`) for the single-value Motoko example, linking
[`motoko/cert-var`](https://github.com/dfinity/examples/tree/master/motoko/cert-var).
- **Root key guidance** now follows the `ic_env` cookie, or `icp network
status --json` in Node, with the same serving-network caveat as #405.
- **Certified assets** point to the `static-site` skill instead.

<details>
<summary>Verification (local network)</summary>

| Check | Result |
|---|---|
| Rust KV (`ic-certification` 4 / `ic-cdk` 0.20), Motoko `CertTree` |
present → value; missing → proof of absence; tampered value → throws;
other key's witness → `Unknown`, throws |
| `@icp-sdk/bindgen` 0.4 actor passed straight to the helper (Motoko
`CertTree`) | present → value; absent key (`value` is `undefined`) →
`null`; tampered or dropped value → throws |
| Motoko single value, `verifySingleValue` | fresh install and after a
set both verify; tampered value throws |
| HTTP canister through the local gateway | `200 hello`, and a certified
`404` for other paths. A variant serving an uncertified body is rejected
with `backend_response_verification`; certifying without
`IC-CertificateExpression` returns `CertificateExpressionHeaderMissing`
|
| Upgrade without re-setting | Motoko single value still verifies; Rust
verifies with a proof of absence after `post_upgrade` |
| Local gateway, tampered response | rejected on `<id>.localhost`,
served on `<id>.raw.localhost` |
| Mainnet ckBTC ledger `icrc3_get_tip_certificate` |
`verifyCertification` passes; `last_block_index` and `last_block_hash`
are `Found` |

</details>

<details>
<summary>Eval results (new file, all cases with baseline, rerun on the
restructured skill)</summary>

- Case 1, "Adversarial: certified getter traps when called from
icp-cli": WITH 3/3 | WITHOUT 3/3. This is a regression guard for the
`--query` pitfall.
- Case 2, "Frontend witness verification with certificate-verification
4": WITH 4/4 (twice) | WITHOUT 3/4. The baseline returns null regardless
of lookup status; a later baseline run timed out.
- Case 3, "Adversarial: certified queries fail after a Rust canister
upgrade": WITH 3/3 (three runs) | WITHOUT 1/3, 2/3, 1/3. The baseline
claims certified data is reset on upgrade.
- Case 4 (new), "Adversarial: gateway assumed to verify Candid query
calls": WITH 3/3 (twice, the second after the gateway-proxy wording fix)
| WITHOUT 3/3. This is a regression guard for the decision table.
- Case 5, "Adversarial: CertTree declarations in a persistent Motoko
actor": WITH 3/3 (twice) | WITHOUT 2/3. The baseline insists on a
postupgrade hook.
- Case 6, "Adversarial: ic-http-certification header and fallback
errors": WITH 4/4 (three runs) | WITHOUT 4/4, 4/4, 3/4. This is a
regression guard for pitfall 9 and the certified 404.
- Case 7, "Adversarial: certify the initial value of a Motoko certified
variable": WITH 4/4 (twice) | WITHOUT 3/4 (twice). The baseline never
certifies the initial value.
- Case 8 (new), "Adversarial: bindgen returns an empty opt as
undefined": WITH 4/4 (twice) | WITHOUT: no result. The baseline produced
no code in either run (a permission request, then a timeout).
- Triggers: should-trigger 4/4, should-not-trigger 2/2.

</details>

Refs #406 (its `certified-variables` row: the `rootKey` type fix is done
here). The developer-docs guides with the same errors are fixed in
dfinity/developer-docs#408.
@marc0olo
marc0olo merged commit 8f7b701 into main Sep 28, 2026
9 checks passed
@marc0olo
marc0olo deleted the docs/certification-accuracy branch September 28, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants